Assessing Fit of Models with Discrete Proficiency Variable in Educational Assessment. Research Report. RR-04-07

Autor/inn/en	Sinharay, Sandip; Almond, Russell; Yan, Duanli
Institution	Educational Testing Service, Princeton, NJ.
Titel	Assessing Fit of Models with Discrete Proficiency Variable in Educational Assessment. Research Report. RR-04-07
Quelle	(2004), (55 Seiten) PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
Schlagwörter	Statistical Analysis; Epistemology; Educational Assessment; Item Response Theory; Research Reports; Methods Research; Bayesian Statistics; Scoring Rubrics; Context Effect; Educational Diagnosis; Goodness of Fit; Achievement Tests; Item Analysis + Suchen Sie Ihr Suchwort? Statistische Analyse; Erkenntnistheorie; Education; assessment; Bewertungssystem; Item-Response-Theorie; Research report; Forschungsbericht; Methodenforschung; Scoring formulas; Auswertungsbogen; Pedagogical diagnostics; Pädagogische Diagnostik; Achievement test; Achievement; Testing; Test; Tests; Leistungsbeurteilung; Leistungsüberprüfung; Leistung; Testdurchführung; Testen; Itemanalyse
Abstract	Model checking is a crucial part of any statistical analysis. As educators tie models for testing to cognitive theory of the domains, there is a natural tendency to represent participant proficiencies with latent variables representing the presence or absence of the knowledge, skills, and proficiencies to be tested (Mislevy, Almond, Yan, & Steinberg, 2001). Model checking for these models is not straightforward, mainly because traditional X[superscript 2]-type tests do not apply except for assessments with a small number of items. Williamson, Mislevy, and Almond (2000) note a lack of published diagnostic tools for these models. This paper suggests a number of graphics and statistics for diagnosing problems with models with discrete proficiency variables. A small diagnostic assessment first analyzed by Tatsuoka (1990) serves as a test bed for these tools. This work is a continuation of the recent work by Yan, Mislevy, and Almond (2003) on this data set. Two diagnostic tools that prove useful are Bayesian residual plots and an analog of the item characteristic curve (ICC) plots. An X[superscript 2]-type statistic based on the latter plot shows some promise, but more work is required to establish the null distribution of the statistic. On the basis of the identified problems with the model used by Mislevy (1995), the suggested diagnostics are helpful to hypothesize an improved model that seems to fit better. (Contains 5 tables and 12 figures.) (Author).
Anmerkungen	Educational Testing Service. Rosedale Road Mailstop 19R, Princeton, NJ 08541-0001. Tel: 609-921-9000; Fax: 609-734-5410; Web site: http://www.ets.org.
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10